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5 1 . (New) The method of claim 47, wherein the encoding comprises the extensible Markup 
Language (XML) encoding. 

52. (New) The method of claim 47, wherein the encoding comprises the extensible Markup 
Language (XML) encoding. 



53. (New) A method for generating description records from multimedia content, 
comprising: 

identifying multimedia types in multimedia content; 

extracting multimedia objects to generate multimedia object descriptions from the 
multimedia content for at least one of the multimedia types; 

generating, from the multimedia object descriptions, multimedia object hierarchy 
descriptions by object hierarchy construction and extraction processing, for at least one of the 
multimedia types; and 

integrating the multimedia object descriptions and the multimedia object hierarchy 
descriptions to generate at least one description record to represent content embedded within the 
multimedia content. 



REMARKS 

The Office Action indicates that the drawings have been objected by the Draftsperson. 
Attached hereto is a Request for Approval of Corrected Drawings that addresses the concerns 
raised by the Draftsperson. Approval is respectfully requested. 

The Office Action rejects claims 1-10 under 35 U.S. C. § 103 over Sezan and rejects 
claims 1 1-24 under 35 U.S.C. § 103 over Sezan in view of Patel. These rejections are traversed 
as they may be applied to new claims 25-53. 

It is respectfully submitted that neither of the applied references disclose or suggest at 
least generating, from multimedia object descriptions, entity relationship graph descriptions for 
at least one of the multimedia times, as recited in new claim 25 or generating, from the 
multimedia object descriptions, multimedia object hierarchy descriptions by object hierarchy 
construction and extraction processing, for at least one of the multimedia types. Furthermore, 
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neither of the applied references discloses or suggest the integrating step subsequently recited in 
each of these claims. Accordingly, none of the claims of the present application are anticipated 
by or are obvious over the applied references. Applicants request withdrawal of the rejections. 

For at least the above reasons, it is submitted that the application is in condition for 
allowance. Prompt consideration and allowance are solicited. 

Attached hereto is a marked up version of the changes made to the specification and 
claims by the current amendment. The attached page is captioned " Version with Markings to 
Show Changes Made ". 

The Commissioner is hereby authorized to charge payment of any fees associated with 
this communication or credit any overpayment to the deposit account of Kenyon & Kenyon, 
Deposit Account No. 11-0600. 

The Examiner is also invited to contact the undersigned attorney if any communication is 
believed to be helpful in advancing the examination of the present application. 



Respectfully submitted, 



Dated: March 24, 2003 




KENYON & KENYON 
1500 K Street, N.W., Suite 700 
Washington, D.C. 20005-1257 
(202) 220-4275 
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APPENDIX 

Version with Markings to Show Changes Made 
IN THE CLAIMS : 

Please cancel claims 1-24. 

Please add new claims 25-53 as follows: 
25 (New) A method for generating description records from multimedia content, 

comprising: 

identifying multimedia types in multimedia content; 

extracting multimedia objects to generate multimedia object descriptions from the 
multimedia content for at least one of the multimedia types; 

generating, from the multimedia object descriptions, entity relation graph descriptions for 
at least one of the multimedia types; and 

integrating the multimedia object descriptions and the entity relation graph descriptions 
to generate at least one description record to represent content embedded within the multimedia 
content. 

26. (New) The method of claim 25, further comprising generating, from the multimedia 
object descriptions, multimedia object hierarchy descriptions by object hierarchy construction 
and extraction processing, for at least one of the multimedia types. 

27. (New) The method of claim 25, wherein the multimedia types include at least one of 
image, audio, video, synthetic, and text. 

28. (New) The method of claim 25, wherein the extracting of multimedia objects further 
comprises: 

segmenting each multimedia content into segments including content from at least one of 
the multimedia types for the multimedia content; and 

generating at least one feature description for at least one of the segments by feature 
extraction and annotation; 
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wherein the generated multimedia object descriptions comprise the at least one feature 
description for the at least one segment. 

29. (New) The method of claim 28, wherein the segments are selected from the group 
consisting of local segments and global segments. 



30. (New) The method of claim 28, further comprising selecting the at least one feature 
description from the group consisting of media, semantic and temporal features. 

3 1 . (New) The method of claim 29, wherein the media features are further defined by at least 
one feature description selected from the group consisting of data location, scalable 
representation and modality transcoding. 

32. (New) The method of claim 29, wherein the semantic features are further defined by at 
least one feature description selected from the group consisting of keywords, who, what object, 
what action, why, when, where and text annotation. 

33. (New) The method of claim 29, wherein the temporal features are further defined by at 
least one feature description consisting of duration. 

34. (New) The method of claim 28, wherein the extracting of multimedia objects further 
comprises: 

generating media object descriptions from the multimedia segment for one of the 
multimedia types by media object extraction processing; 

generating media object hierarchy descriptions from the generated media object 
descriptions by object hierarchy construction and extraction processing; and 

generating media entity relation graph descriptions from the generated media object 
descriptions by entity relation graph generation processing. 

35. (New) The method of claim 34, wherein generating media object descriptions further 
comprises: 
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segmenting the content of each multimedia type in the multimedia object into segments 
within the multimedia object by media segmentation processing; and 

generating at least one feature description for at least one of the segments by feature 
extraction and annotation; 

wherein the generated media object descriptions comprise the at least one feature 
description for the at least one of the segments. 

36. (New) The method of claim 35, further comprising the step of selecting the at least one 
feature description from the group consisting of media, semantic and temporal. 

37. (New) The method of claim 35, wherein generating media object hierarchy descriptions 
generates media object hierarchy descriptions of the media object descriptions based on media 
feature relationships of media objects represented by the media object descriptions. 

38. (New) The method of claim 35, wherein generating media object hierarchy descriptions 
generates semantic object hierarchy descriptions of the media object descriptions based on 
semantic feature relationships of media objects represented by the media object descriptions. 

39. (New) The method of claim 35, wherein generating media object hierarchy descriptions 
generates temporal object hierarchy descriptions of the media object descriptions based on 
temporal feature relationships of media objects represented by the media object descriptions. 

40. (New) The method of claim 35, wherein generating media object hierarchy descriptions 
generates media object hierarchy descriptions of the media object descriptions based on 
relationships of media objects represented by the media object descriptions, and wherein the 
relationships are selected from the group consisting of media feature relationships, semantic 
feature relationships, temporal feature relationships, and spatial feature relationships. 

41 . (New) The method of claim 35, wherein generating media entity relation graph 
descriptions generates entity relation graph descriptions of the media object descriptions based 
on relationships of media objects represented by the media object descriptions, wherein the 
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relationships are selected from the group consisting of media feature relationships, semantic 
feature relationships, temporal feature relationships and spatial feature relationships. 

42. (New) The method of claim 26, wherein generating multimedia object hierarchy 
descriptions generates multimedia object hierarchy descriptions of the multimedia object 
descriptions based on media feature relationships of multimedia objects represented by lie 
multimedia object descriptions. 

43. (New) The method of claim 26, wherein generating multimedia object hierarchy 
descriptions generates semantic object hierarchy descriptions of the multimedia object 
descriptions based on semantic feature relationships of multimedia objects represented by the 
multimedia object descriptions. 

44. (New) The method of claim 26, wherein generating multimedia object hierarchy 
descriptions generates temporal object hierarchy descriptions of the multimedia object 
descriptions based on temporal feature relationships of multimedia objects represented by the 
multimedia object descriptions. 

45. (New) The method of claim 26, wherein generating multimedia object hierarchy 
descriptions generates multimedia object hierarchy descriptions of the multimedia object 
descriptions based on relationships of multimedia objects represented by the multimedia object 
descriptions, wherein the relationships are selected from the group consisting of media feature 
relationships, semantic feature relationships, temporal feature relationships and spatial feature 
relationships. 

46. (New) The method of claim 25, wherein generating entity relation graph descriptions 
generates the entity relation graph descriptions of the multimedia object descriptions based on 
relationships of multimedia objects represented by the multimedia object descriptions, wherein 
the relationships are selected from the group consisting of media feature relationships, semantic 
feature relationships, temporal feature relationships and spatial feature relationships. 
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47. (New) The method of claim 25, further comprising receiving and encoding the 
multimedia object descriptions into encoded description information, and storing the encoded 
description information as the at least one description record. 

48. (New) The method of claim 26, further comprising combining the multimedia object 
description^ the multimedia object hierarchy descriptions, arid the entity relation graph 
description to form a multimedia description, and receiving and encoding the multimedia 
description into encoded description information, and storing the encoded description 
information as the at least one description record. 

49. (New) The method of claim 47, wherein the encoding comprises binary encoding. 

50. (New) The method of claim 48, wherein the encoding comprises binary encoding. 

5 1 . (New) The method of claim 47, wherein the encoding comprises the extensible Markup 
Language (XML) encoding. 

52. (New) The method of claim 47, wherein the encoding comprises the extensible Markup 
Language (XML) encoding. 

53. (New) A method for generating description records from multimedia content, 
comprising: 

identifying multimedia types in multimedia content; 

extracting multimedia objects to generate multimedia object descriptions from the 
multimedia content for at least one of the multimedia types; 

generating, from the multimedia object descriptions, multimedia object hierarchy 
descriptions by object hierarchy construction and extraction processing, for at least one of the 
multimedia types; and 

integrating the multimedia object descriptions and the multimedia object hierarchy 
descriptions to generate at least one description record to represent content embedded within the 
multimedia content. 
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